"OpenAI said it has paused training of its latest AI models," reports the Associated Press, "as reports of AI agents going rogue mount." The decision to halt development came just hours after the company disclosed Friday that it was reviewing several incidents from the summer in which OpenAI agents searching federal government websites acted in unexpected ways beyond what was asked of them while gathering and distributing information... OpenAI said in a statement that it will resume training "only when we are confident that we have additional safeguards" in place, adding that it expects it will have to "hit pause" again as AI develops and other issues emerge... It is the second time in three months that OpenAI has halted development of its models. The first came in July after disclosure of a cyberattack targeting AI startup Hugging Face, a now notorious incident that raised fears the industry was losing control. OpenAI "also said it had notified dozens of third parties about improper activity," reports Reuters: As of mid-September, one person briefed on the matter estimated that OpenAI had found roughly two dozen incidents of its agents acting in undesirable ways. But the number has continued rising as OpenAI teams sift through internal logs of the agents' activities and find previously unknown cases, the two people close to the company said... OpenAI has acknowledged a general need for more transparency around rogue AI behavior... Even so, two people familiar with OpenAI's investigation into its agents' activity described it as locked down and shaped by company lawyers. The process has been unusually compartmentalized for a company that some former employees say was more open about these issues in the past, the people said. Roughly 100 people were in some way involved in the process to understand the Hugging Face hack, three people briefed on the matter said. During that process, evidence of other incidents surfaced. Reuters has previously reported that OpenAI investigators looking into the Hugging Face breach were discouraged by the company's lawyers from expanding the scope of the investigation to include other incidents. OpenAI said its lawyers did not discourage deeper investigation. Many incidents have been uncovered by outside researchers rather than OpenAI directly. In several episodes, the agents took problematic actions that went unnoticed by the company for months. Meanwhile, Axios reports that Anthropic's Claude Opus 5.5 model "sought to escape a sandbox — a secure testing environment — in 1.5% of test runs, though the company emphasized that these were adversarial experiments where a task couldn't be solved without escaping the sandbox." Anthropic points out that those tests were run "without the additional safeguards we apply in production". But they acknowledged that then Claude Opus 5.5 "when given apparent credentials to a public package registry in a simulated security exercise, took potentially harmful actions in roughly half of cases. Very rarely, pre-release snapshots produced and acted on spontaneous malicious tool calls, and during training some snapshots concealed actions from an automated grader." Claude Opus 5.5 "showed less misaligned behavior and less cooperation with misuse than any other recent Claude model on nearly all measures," Anthropic adds, and "took overeager or destructive actions less than any other model we tested." But Axios makes an interesting estimate about that 1.5% of test runs (without safeguards). "Anthropic and other companies conduct hundreds of thousands of test runs on their models, or more, sources said. That means even a small percentage of misaligned behavior can still amount to tens of thousands of incidents in which the models behaved in unexpected, sometimes troubling ways." The sheer number of incidents, which occurred in recent months in internal testing and the real world, indicates that the problem is orders of magnitude more complex than what is publicly known. The findings, which are surfacing as part of internal work to assess models and in investigations at both companies into model behavior, raise questions about whether either company — or any top model-maker — is currently capable of establishing complete control over their technology. The episodes include bypassing guardrails, creating message boards, escaping sandboxes, website hijacking, self-prompting or seeking to bypass monitors, sources said. They occurred in internal testing and in the real world, and many have yet to become public as security researchers continue to investigate, sources said... Some at OpenAI see Hugging Face as a one-off, with disclosures about future incidents likely to be less severe due to improved controls and the unusual nature of the testing they conducted, which involved an unreleased model, sources told Axios. AI security researchers agree that there are simple fixes that will help AI companies avoid aspects of what made the Hugging Face episode appear so dangerous to outsiders. Other AI executives and safety researchers, however, cautioned that they have limited confidence that AI companies will be able to prevent all problematic model behavior... It's not about how damaging each individual instance was, Connor Leahy, AI researcher and executive director at ControlAI told Axios. The "crazy thing," he said, is that these instances involve "autonomous systems doing things they were told not to do," potentially including crimes.

Read more of this story at Slashdot.

With the right software, you can run Android apps on your Windows PC. Here's why you may want to and how to get it set up on your own computer.

Could this push solar cell efficiency beyond the theoretical 33% limit? Interesting Engineering reports: Researchers at the University of Groningen in the Netherlands found that tin-based perovskite solar cells can slow heat loss from high-energy "hot electrons..." When sunlight strikes a panel, photons jump-start electrons into action. The most energetic photons create super-charged hot electrons... [but] in fractions of a trillionth of a second, these high-energy particles rapidly cool, dumping their bonus energy as waste heat before ever leaving the solar cell... In collaboration with Maria Antonietta Loi, professor of Photophysics and Optoelectronics, the team created an experimental setup. Using a specialized solar cell material called tin-based perovskite, Loi's lab performed a feat many thought impossible: she slowed the heat loss down by a factor of 1,000. Suddenly, the extra energy lingered for nanoseconds instead of vanishing in picoseconds... To solve the puzzle, Koster and PhD student Tim Faber built digital simulations to peel back the quantum layers. And discovered a surprising double-action mechanism at work... The simulations matched the exact nanosecond delay observed in the lab... These specialized materials could be used to build a new generation of super-efficient solar cells. Tin-based metal halide perovskites are non-toxic, eco-friendly crystalline materials for high-performance solar energy conversion... The material possesses an unusually low electron mass. As a result, electric charges move quickly and retain extra thermal energy for extended periods. This combination of broad light absorption, efficient charge movement, and prolonged energy retention makes these materials prime candidates for next-generation solar panels. "There are many other questions that still need answers," the team said in their announcement, "but in theory, this discovery could allow the creation of more efficient solar cells, beyond the theoretical limit of 33 percent." Thanks to long-time Slashdot reader fahrbot-bot for sharing the article.

Read more of this story at Slashdot.

An old netbook can automate the devices that otherwise aren't interested in talking to one another.

A customer reported that their employees were randomly getting “memory write errors”.

This was their way of interpreting the error message

The instruction at “XX” referenced memory at “YY”. The memory could not be “written”.

Okay, so what we have here is an access violation.

The strange thing was that this access violation was happening across multiple unrelated programs, rather than all occurring in a single program or family of programs. So there is some sort of broader problem here, rather than just a single buggy program.

Opening one of the crash dumps shows this:

eax=0013d354 ebx=0049a000 ecx=009e9a9f edx=03111160 esi=009e9aa0 edi=009e9aa0
eip=009e9aa7 esp=003cfda4 ebp=003cfdb0 iopl=0         nv up ei pl nz ac pe cy
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010217
WerFault!wmainCRTStartup+0x7:
009e9aa7 0000            add     byte ptr [eax],al          ds:002b:0013d354=??
0:000>

That add byte ptr [eax], al should immediately tell you that we are not executing valid code: It is the instruction that you get if you try to execute zeroes. You can see the zeroes in the second column.

All of the crash dumps look like this, just with different process names.

Let’s disassemble from the start of the function to see how we got here.

0:000> u .-7
WerFault!wmainCRTStartup:
009e9aa0 90              nop
009e9aa1 49              dec     ecx
009e9aa2 ba6011f102      mov     edx,2F11160h
009e9aa7 0000            add     byte ptr [eax],al ← died here
009e9aa9 0000            add     byte ptr [eax],al
009e9aab 41              inc     ecx
009e9aac ffe2            jmp     edx
009e9aae cc              int     3

This doesn’t look like the proper start of a function.

I mean, one clue is that it starts with a single-byte nop, rather than a mov edi, edi, as is customary for x86-32 code.¹

And then of course there is the chunk of four 00 bytes in the middle of the instruction stream.

What I thought was interesting is that if you take out those four 00 bytes, then the dec ecx and inc ecx cancel out, and what’s left looks like a detour: It loads an absolute address into edx and then jumps to it.

This looks to me like a failed attempt to detour the function. The next step is to try to figure out what they were trying to do.

Well, it looks like it’s trying to detour to a function at 0x02f11160, so let’s see what the debugger can tell us about that.

0:000> !address 0x2f11160

Usage:              <unknown>
Base address:       02f11000
End address:        02f12000
Region Size:        00001000 (4.000 kB)
State:              00001000 MEM_COMMIT
Protect:            00000020 PAGE_EXECUTE_READ
Type:               00020000 MEM_PRIVATE
Allocation Base:    02f10000
Allocation Protect: 00000004 PAGE_READWRITE

So this is a mystery 4KB allocation of executable memory.

Maybe there are some interesting strings in that memory block.

0:000> !strings 02f11000 02f12000
02f11080 --------
02f11148 ----------------
02f11472 C:\Program Files\Common Files\Contoso\injcore.dll
02f11545 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII

Okay, well, that path to a DLL kind of catches them red-handed. I bet the “inj” stands for “injection”. But what were they trying to do?

I figured, “Hm, the extra four zero bytes come right after the constant they were trying to load, so if I change the mov edx to a mov rdx, it would be the upper half of a 64-bit constant, and then this would look okay again.

Now, this is a 32-bit process (evidenced by the 32-bit instruction pointer), so there is no mov rdx instruction. That instruction requires a 64-bit process.

But wait, what if they got confused and thought it was a 64-bit process?

Let’s disassemble these bytes as if they had been injected into a 64-bit process.

The way I do this is to load up a sacrificial 64-bit debug session and just patch into it the bytes that I want to study. For poetic irony, I will load the 64-bit WerFault.exe into the debugger as a dump file.

C:\> windbgx -z C:\Windows\System32\WerFault.exe

Executable search path is: 
ModLoad: 00000001`40000000 00000001`400a1000   C:\Windows\System32\WerFault.exe
WerFault!wmainCRTStartup:
00000001`40002480 sub     rsp,28h
0:000> eb . 90 49 ba 60 11 f1 02 00 00 00 00 41 ff e2 cc
0:000> u .
WerFault!wmainCRTStartup
00000001`40002480 nop
00000001`40002481 mov     r10,2F11160h
00000001`4000248b jmp     r10
00000001`4000248e int     3

Okay, now it makes much more sense. This is a 64-bit detour that loads an absolute jump target into a 64-bit register (r10) and then jumps to it.

They injected 64-bit code into a 32-bit process!

This also explains why the crashes are sporadic: The customer’s employees run 64-bit processes most of the time, but on occasion, something will run a 32-bit process, and those are the ones that are crashing.

Upon further discussion with the customer, we learned that Contoso is an anti-malware program that they use. We advised them to disable it temporarily to confirm that it was the source of the problem, but they didn’t want to disable their anti-malware software.

Okay, so we advised them to check with the vendor to see if an update is available. They were resistant to changing their anti-malware software without first putting it through their internal validation. They considered this a Windows problem, and they demanded a Windows solution.

We are still working to convince the customer that they need to re-evaluate their anti-malware software.²

¹ Even if this were a 64-bit process, Windows components don’t begin functions with a single-byte nop or any other single-byte instruction. It would use a two-byte nop if it uses one at all.

² This is a downside of communicating with the customer through a customer liaison: My colleague explained that relaying this level of detail through a customer liaison who is not sufficiently technical to be familar with debugging puts us at a disadvantage because the liaison can’t stand up to the customer pushback. This is a case where we may have to let the engineers talk directly to the customer.

The post Debugging walkthrough: Access violation on nonsense instruction, episode 3 appeared first on The Old New Thing.

Heidi Overton, President Trump's nominee to lead the Food and Drug Administration, went before the Senate health committee Thursday, facing questions about her stances on vaccines, flavored vapes, birth control, and abortion, among other issues. She spent nearly two hours stonewalling senators, even on softball questions.

Overton, a physician with a doctoral degree in clinical investigation, is currently the deputy assistant to the president for domestic policy. She previously worked at a conservative think tank and has taken a clear stance opposing abortion, raising obvious questions about her objectivity. The FDA is currently reviewing the safety of the abortion pill, mifepristone, despite extensive data showing it is extremely safe.

Overton also notably stood behind President Trump at a press conference in August during which Trump falsely called the measles, mumps, and rubella (MMR) vaccine "quite lethal." He further claimed that vaccine doses for infants are "like the size of a soda bottle," and called for the MMR vaccine to be split into three separate immunizations, despite no evidence of benefit from such a split and no such individual immunizations being available in the US.

Vaccine contradictions

In the hearing, Overton directly contradicted Trump on several occasions, saying at various times that "the MMR vaccine is not lethal" and that "the MMR vaccine is safe and effective."

Still, when Sen. Maggie Hassan (D-NH), asked "Do you think it's possible for the president to ever be wrong about something?" Overton dodged. Instead of answering the question, she began spouting a non-answer that started with: "I've had the privilege of working for the president for over a year and a half and have had robust discussions with him."

Hassan cut her off, pressing: "Is it possible for him to be wrong?" But again, Overton was evasive.

Hassan asked her if she thought the MMR vaccine might be lethal. Again, Overton said it was safe and effective. "So," Hassan asked,"was the president wrong when he suggested that the combined measles, mumps, and rubella vaccine may be lethal?"

"Senator, the president asks questions and wants the best outcomes for the American people right now."

Hassan kept pressing, switching to Trump's false claim that Tylenol taken during pregnancy is linked to autism in children. "Was he wrong when he said that? Yes or no?" Again, Overton didn't answer, only saying "There is no current evidence of a causal link between Tylenol and autism."

"I wish it wasn't so difficult for members of the administration to just plainly say that the president's wrong," Hassan said. "And it's really concerning to me that you are equivocating on it here."

Bad options

Sen. Lisa Blunt-Rochester (D-Del.) also asked about the vaccine claims Trump made in August, noting that reports suggested it was Overton herself who suggested to Trump that the combination vaccine could be split into multiple shots. Even if there was reason to think splitting the vaccine would improve safety and/or efficacy, it would take huge investments and years of trials to get the individual vaccines approved in the US.

"So obviously," Blunt-Rochester asked, "you think these were good options, correct?"

Overton replied that, in her current role, her job is to give Trump options.

"You wouldn't have provided bad options, right?"

But Overton again wouldn't answer Blunt-Rochester's question. "This is just a simple question," Blunt-Rochester interjected. "This, I thought, was the easy question." Blunt-Rochester went on to ask Overton if she reviewed data about splitting the MMR vaccine or if current doses were the size of a soda bottle, but again she dodged the questions.

Flavored vapes

The FDA has been without a commissioner since May, when Marty Makary resigned from the role after resisting Trump's pressure to approve flavored vapes. The FDA issued a policy allowing their sale in early May, despite bipartisan opposition to the products, which are largely seen at being marketed to youth.

Sen. Chris Murphy (D-Conn.) laid out a timeline of how it happened, saying:

On April 30th, it's a Thursday afternoon, Reynolds, one of the biggest tobacco companies in the country, makes an $8 million donation to the president. Two days later on Saturday, because of that donation, they have lunch with the president and they tell him that they want a change in FDA rules, especially regarding flavored vapes. The president calls the FDA commissioner, he calls the secretary of [the health department], and three days later on Tuesday, new guidance is issued that paves the way to the sale of flavored vapes.

Murphy asked Overton if she saw a connection between the donation and the policy change, and if that's how she thinks the FDA should work. "I reject the framing of that question," Overton replied, adding that Trump is "committed to the best possible outcomes for American."

Sen. Tim Kaine (D-Va) pushed Overton on upcoming plans that the Trump administration has to further ease regulation of tobacco products, including fruity vapes. Kaine asked what Overton, a key domestic policy advisor, knew about the upcoming changes. But Overton would only say that she was "not involved in those discussions."

"Dr. Overton, now come on, don't play games with me," Kaine said. "I didn't ask whether you were involved in them. I asked, are you aware ... are you aware of discussions that is going on about whether the FDA will roll back tobacco safety regulations? And that's an easy [question]. You're either aware of them or you're not."

After again declining to answer the question, Kaine gave up, saying "Let the record reflect that the witness refused to answer the question. I yield."

"Pretty good job"

On the topic of birth control and abortion, Overton was equally evasive, refusing to say whether she thinks IUDs (intrauterine devices) are a form of abortion or whether women are polluting the water supply by taking birth control pills—both things anti-abortion groups claim.

Even for friendly questions, Overton refused to answer things directly. Sen. Ashley Moody (R-Fla.) asked about illegal, imported vapes, something Overton mentioned wanting to crack down on in her opening statement. Moody asked a softball question, simply getting Overton to confirm the FDA's current rules for vaping products, which allow products to be sold while applications for FDA approvals are pending.

"If someone files an application, they can sell, right?" Moody asked multiple times.

But Overton didn't answer the question and Moody became frustrated, exclaiming at one point: "I'm just trying to find out what the law is!"

Halfway through the hearing, Murphy summed it up succinctly, telling Overton: "You've done a pretty good job of avoiding answering questions."

Read full article

Comments